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|| OFFICE OF THE ASSISTANT SECRETARY OF DEFENSE | 
"0 S WASHINGTON. D. €. 20303 FAL ET LZ 6 py 75. 


ca ae . ‘spond May 1975 ; 


on 


a = Deputy to the DCI for the 
1, Intelligence Community ©. 
“Washington, D. C, 20505 


O° Dear L1G Wilsons ne ne ee Baa 
a 5 = % New Business for 20 May Meeting of the NSCIC Working Group 
: i “Ambassador Ellsworth requested that I forward the following — 


ae comments and suggestions regarding the 20 May meeting of the NSCIC 
_ Working Group, and possible new business for future meetings. 


i A, Status of Committees on Consumer Evaluation and Coneumer 
2 _Guidance _ : = 
: - These committees could be highly useful. The committee for 
*"Consumer Evaluationiof Current Intelligonce Products" represents an 
‘important step forward in developing the kind of detailed consumer evaluation 
. | of the intelligence product which is essential to support the NSCIC, and to 
. - provide the Intelligence Community with a user view of its output. Sucha - 
.) committee could not only establish suitable procedures for obtaining user 
Views from different agencies, but serve as a mechanism fox evaluating - 
“major products. The "Consumer Guidance"! committee also seems useful, 
.. (Sand could serve a valuable purpose in giving the Intelligence Community a 
‘better view of the needs of different agencies for given intelligence products, 
[4 and for substantive detail in providing intelligence on given subjects. This 
«committee might be particularly ueeful in reviewing the scheduling of major - 
intelligence products relative to the KiQs, Objectives, and needs of given 


". gonsumers, 3 ‘ ia 

ae 08 ‘A proposal: for setting up such committees was originally 
_..8cheduled to be distributed to members of the NSCIC Working Group by 
US April. It would be helpful to have auch a proposal as soon as possible 
- 80 that it can be discussed at the next meeting of, the Working Group. 
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2. ‘Date of the Next Meeting 


nt would be helpful if the next meeting of the NSGIC/ WG ebuid. 
be timed so that members had an opportunity to review NIAM 35/36 before 
..ythe meeting. We would like to discuse this NIAM, and to present a Grete 
ae “Meritique" z as Eeauestee in your note of 11 March, 1975. 


eas “Necdtonan Annotated Bibliogxaphy of NIEs, SNIKs, NIAMs, 
an: and NSSM. Support Products 
a 


. ‘Ie became sonasent in surveying various components of OSD that 
.. national intelligence products are not being properly utilized because no 

_... adequate bibliography exists of such documents. We have looked into this 
Hee Be REOv SE) and the situation can be Bue T TESS as follows: 


fs ee We Rags surveyed OSD views towards user teview of NIEs, 

# SNIEs, and NIAMs. We strongly endorse this idea, We have selected NIAM 
35/36 for inital review and we have determined other documents for future 
ae review. We are Pevelopang) methodology for user review. 


i oc "We found: dn anise. thia survey that we have a major problem 
-'» in OSD ee no useful or timely annotated bibliography exists of NIEs, 
/* NIAMs, and SNIEs, The overall impact of the national]. intelligence effort 

eae | Sopalies reduced hy poor information flow caused by lack of a timely 

“wai to survey overall intelligence effort, a lack of staff knowledge of 

staat onal level products, and a lack of pany way to learn what has been done 

a in the Cae {8 

; aT The DIA. USIB cers tried to create an index of NIE‘s, SNtke, ; 

‘and NIAMe for us. We found its holdings had serious gaps; and were some. 

_/ times out of date. 3 | | : 

| 

ae -- .We forte some Hedalicta not getting to Rey cities in. PARE, A 

cod “ISA (P/P) and DDR&E because they only need these for specific studies 

es ae have no timely or comprelennive index to consult. 


e e- There ‘used to be a comprehensive index of NIEs, SNIEs,. tena 
ue NIAMs until 1964. It was not annotated and was not organized into functional 
zn ' categories, 


i 
‘ =« There ‘is now eae: an annual index of NIEs, SNIEs, “and NIAMs 
; which comeg out ed mace months eee and of year er It does not 


een een om at 


| published duri ing 1974 g 1974, 6 Mar ch 1975). 


a he Fa Bn is eee, ET Nor 
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as ae : Approved For’ Releave’ 20009103 wn SEBEL 7 sed 
es DIA does have an are tient semi-annual. and annotated index 
*: of DIiEks, SDIEs, DIAMs, and DIEMs' (DIA, “Estimates Publications me List," 
>... | $22008/DE-X (S) published in last six months. It also lists all documents 
"Published in last five years, 7 rat 
‘This lack of a proper and timely ainotated bibliography of national 
"intelligence documents should be corrected as soon as possible. Accord- 
“ingly, we request that the IC Staff take ‘the aii steps to solve. the 
Pre ‘oblem: ; on 
ae lee Immediately add an auatnatent ot the DCI "Status of Work 
‘in Beopeeee Report! which lists all products produced since the pszies 
“Govered in USIB-D-17, 15, 


rae Prepare a seats annotated uibtibacanhy of the Hype aevetsecd 
an DIA in §-2008/DE-X(S) for the period since USIB-D-17, ‘1s and present 
oe _ thie at ne next meeting | of the NSGIC/WG, _ ‘ 


aa oe _ ae Begin developrient of a ‘sommpeeheneive Sinotated pibliodzasiy 
“of the national intelligence product. This should include all documents not 
. ° updated in the last five years. It should include a future production achedule 
- for the comming 12 months. It should he updated semi-annually, and it 
a ae “ghould have a specific Meting: of ell regularly scheduled products not mpiated 
_ fering the last year, ; 


ome aS ; 
iki, es This document will not only be a major step forward in solving’ 
er the information flow problem, but provide the ner and MeCTSh WG with: 


‘an overview of the national predact effort, - 


oe a : 4, The FY 76 Key. Intelligence Questions 


os OSD is developing a package of user comments on the FY 76 
 KIQs, The main result of sur veying users has, however, been to indicate 
that many do not feel that the KIQs are important enough to merit detailed - 
_teview. The FY 75 KIQs are felt to be sa general that they do not provide 
Meaningful guidance, Further, the KiQ strategy reports suffer from the 
‘\6ame problem, They do: not address what substantive issues or data needs 
‘should be addressed in answering the KIQ, and they do not evaluate whether 
“the products listed actually answer the KIQ, Instead, they associate all 

jo “Belated output to a given Hs and make the valume of related output the 

Measure of success. 
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: —- H a ae : : 
Es SB tATiS crlovileate a coordinated DoD > input for the FY 76 KiQs 
“2° dn the form. of Defense Key Intelligence Questions. Unlike the present 
. KlQs, thease provide (a) a rationale for asking the question, (b) a detailed 
, ‘sting of the substantive data needed to answer the question in the form 
of "Sub-DK1Qs,'"' and (c)'an indication of priorities. Such a format is 
| distinctly preferable to that of the FY 75 KIQs. It is particulaxly important 
that the FY 76 KIQs listiin detail the substantive areas where improved 
| "dntelligence is required’ ‘to answer the question. Accordingly, it would be 
|< ‘ugeful if the IC,Staff would examine the format of the DKIQs as soon as 
-. they are submitted by DIA, and present ideas for improving the format | 
a sof the IgE at the next meeting of the. NSCIC/WG. 


ee It would also be useful if the IG Staff could scogent a state. 
a paper with an evaluation of. the effectiveness of the KIQ system, and a 
- glearer explanation of how the IRs are derived, This should contain the | 
following information: f 


a e+ The “ows of the individual NIOs and a pitas of ae 
agers to seo how valuable they felt the ae 75 ae were as guidance to 
the community. ‘ . oe jah, 


ag te a ° 7 ee A detailed analysis of how many NIEs, [NIAMs, SNIEs 

’ es canetated since the issuing of FY 75 KIQs were specifically in response 
ote KIQs or were in response to other forme of specific tasking. Do the . 
‘KIQs zeety guide the community? — — 

a we A déseriplion of how various agency inputea are ‘combined - 

ee into KIQs, of who choses which inputs are made into KIQs, and of how 

ee these choices are made. ; Bott | 


sii S, the KIQ FY:75 Evaluation } Performance Report’ 


dit is not clear that the presént structure of the KEP is what is 
oe required, or that the effort involved id justified... The detailed formats 
- ineludéd in the KEP questionaire may be of uncertain value, and could. 
result in the KEP effort being Little more than a "paper chase."' It would 
be useful if the IC Staff could prepare an analysis of the KEY BEGCEEe which 
deale with the following issues. , 


7 Which dtaffs used the K1Q6? Were the FY 75 KIQs adequate? 


bs e+ What kind of user evlew of KIQ performance would be most 
ne useful? Should the NIQs be tasked with surveying users in filling out an 
ee ages KEP? 


2 
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ae ” . «+ Should the NIOs be involved in this kind of resource 

analysis Eiates selvitg? ; 

me | as Is an updated KIQ Stratos Report really all the review 
be ‘effort zeae ed? Is the KEP worth its cost? 


ee A ee Are the: KIO epecific and detailed enough to justify and 
ah pormét detailed audit treile of the type the KEP system calle for? - 


os ‘Se ‘Does the KEP system fox ‘relating levels of stfoet costs, _ 
and collection means to: epocitic Hie produce meaningful sate? 
eae ; ‘Would saraalivg eecunlauee. including picatet eee 

: and. user reactions, produce more valuable results? 


“The ys amily, of intelligence Documents! 


ae 2 It nay be helpful to Siac in more detail the comments Mr.” 

Ellsworth made on this document. si ads draft should be eepaceey tor 
is ’ Deseribe who writes the ieucent: how it is written, whose. 

: views and eee aro solicited, and how user views are surveyed and taken - 

_ dato account. ae Z ; oe ae Ae 


Tort en ee Doscribé the planning <eyele involved, and how. it relates 
a: to ‘the 2 budget cycle and Seer on. saucer resource requirements, 
Ve se oe “= Describé how the Sauer shes are used in the Intelligence 
tet mapas tnety impact, and who they influence, . 
a tt would alas be helpful if: ‘the IC Staff could survey user reactions 
'¢9 the value and success of the current documents, and if the informal 
“eamments of the individual NiOs on the value and success of each document 
could be surveyed and phogented to members. of the Working ‘Group in the 
a near future, 
7% The P: Problem: of  Hupressing U Uncer tatty in _ Intelligence 
. Egtimates, o_o 


cae The importance of this IC Staft afer SGaHDGE be over- = epapineiaad’ 
ue “and it must not be generated in the form of a "broad brush'' treatment, or 
- glass over the problems involved. A brief review of national intelligence 
. aaa indicates that farr too many: | : 
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ee ee a ee  DEUKEL NOFORN | 
Mei oe _Aovroved For r Relegse 2 2001/09/63 , CIARDP84B00506R000100120014- 7 
2 ee Fail to éxplicitly state the limitations in the iNtalligance: 
“iweltatils or the mcthods of anal ysis used and present them in summary 
form for quick review. . In many cases; coven those limitations that are 
Heted are buried in the text or ated only in footmotes or annexes. 


rf | «~ Fail to sectioiiy discribe the methodology used hen hard” 

_ intelligence - ta is la eking. : | 
sh Fail to clearly distinguish reasonable conclusions or logical 
“views from Sanciueteue .based on intelligence. . . 
aes -- Fail to describe the uncertainty present in given data, 
ae mothods of paelyetey: or, conclusions, ; 


bee 


Cons os le Baal to sanauiey undertatity, fail. to describe the method 
used | to quantity uncertainty, ox fail.‘to show a range of SOME TA: 


os Fail to. list the explicit intelligence data on. which a conn — 
clusion is based, and provide only a broad or vague velonelcs : 


Ny 
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i 
yg o ‘The IC Staff paper should specifically address these mo eiee 
oIé should also survey the last year's output, and specifically list those docu-+ 
“ments where it is felt such problems did not occur, It should not simply - | 
_.* gurvey the methods being used in the Community. It should specifically - | 
Bae . identify the specific products to which they. were applied, and it should 
ree indicate the Percenivee of total anual output in witch such  sthodelsese a 
a were eer ec ak 
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oes ay Anthony H, Cordesman 

Coe oy ona. | Chief, Product Evaluation Division 
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